The invention relates generally to a d-dimensional signal enhancement method, apparatus and computer program, and in particular to a method, apparatus and computer program useful for enhancing d-dimensional signals. Sounds are examples of one-dimensional signals, images are examples of two-dimensional signals. Three-dimensional signals may correspond to video sequences or to three-dimensional block of data such as seismic data or medical imaging data. Signal enhancement includes enhancing the signal resolution with a super-resolution process and enhancing the signal values with a signal restoration. Enhancement is different from compression which either maintains or degrades the signal values in order to construct a compact binary code.
Signal restoration is a process that takes an input digital signal and improves the signal values by removing noise components and by suppressing existing distortions introduced by some prior transformation or degradation process such as a blurring or a signal compression process. Sharpening the signal by removing a blur is a signal restoration as well as a removal of compression artifacts or any additive noise. In seismic signal processing, restoring the reflectivity values from measured seismic data is an another example of signal restoration. Signal super-resolution is a process that improves the signal resolution by adding new sample values to the input signal. This is performed by refining the signal sampling grid and estimating the new signal sample values. An image zooming that increases the image size by refining the image sampling grid is a super-resolution process. For video sequences, deinterlacing processes are also super-resolution processes. Interlaced images include only either the odd or the even rows of the full image. A deinterlacing is a super-resolution process which refines the sampling grid by adding the missing even or odd rows of samples for each interlaced image. Super-resolution for video sequences may also be a scaling that increases the size of each spatial image by adding more pixels (samples). Conversion of standard television images (PAL or NTSC) to high definition television standard includes such a scaling. A restoration process often needs to be integrated with a super-resolution process to improve the signal resolution while removing noises and distortions.
Signal restoration and super-resolution take advantage of a signal regularity. If the input signal values have regular variations, additive noise can be suppressed with a local averaging. Similarly, new sample values can be calculated with a local interpolation. Linear estimators as well as linear interpolations have been thoroughly studied and used in signal processing. Yet, these processes introduce large errors in the neighborhood of sharp transitions such as image edges. Non-linear restoration processes have been introduced to adapt the local averaging to the local signal regularity. Redundant wavelet thresholding processes are examples of such efficient non-linear estimators as described in D. Donoho and I. Johnstone, “Ideal spatial adaptation via wavelet shrinkage,” Biometrika, vol. 81, pp. 425-455, December 1994. For multidimensional signals, wavelet procedures do not take advantage of existing geometrical signal regularity. Sparse spike inversion is another example of non-linear signal restoration procedure used for seismic signal processing to restore the spikes of the reflectivity by minimizing the 11 norm of the restored signal. However, such procedures do not either take advantage of existing geometric signal regularity in the seismic data.
A multidimensional signal may have locally a sharp transition in one-direction while varying regularly in another direction. Taking advantage of such anisotropic geometric regularity is at the core of geometric enhancement processes. For video sequences, motion compensation processes are examples of anisotropic geometric regularization in time. Linear restoration processes with motion compensation have been used to suppress noise from video sequences, with a time regularization performed with a linear filter. Such a linear filter blurs the sharp time transitions of the video. To limit this blurring the linear filter often has an impulse response with a fast decay which limits the averaging and hence limits the noise removal. To remove more noise by taking advantage of the signal regularity over large domains while preserving sharp transitions requires to use an adaptive non-linear regularization. Such adaptive restorations can be implemented with wavelet transforms.
Motion compensated wavelet transforms have been used to compress video sequences with lifting transformations along motion threads as in U.S. Pat. No. 6,782,051. Motion threads are obtained by following the motion in time. However, the purpose of these lifting schemes is to compress and not to enhance the signal. As a consequence critically sampled lifting schemes are used, to decompose the signal with a non-redundant lifting. Redundant lifting that compute oversampled transformations are more efficient for signal enhancement.
To restore sharp signal transitions, a super-resolution process can also use anisotropic signal geometric regularity. Adaptive interpolations can be calculated in the directions in which